Search for: All records

Task arithmetic refers to editing the pre-trained model by adding a weighted sum of task vectors, each of which is the weight update from the pre-trained model to fine-tuned models for certain tasks. This approach recently gained attention as a computationally efficient inference method for model editing, e.g., multi-task learning, forgetting, and out-of-domain generalization capabilities. However, the theoretical understanding of why task vectors can execute various conceptual operations remains limited, due to the highly non-convexity of training Transformer-based models. To the best of our knowledge, this paper provides the first theoretical characterization of the generalization guarantees of task vector methods on nonlinear Transformers. We consider a conceptual learning setting, where each task is a binary classification problem based on a discriminative pattern. We theoretically prove the effectiveness of task addition in simultaneously learning a set of irrelevant or aligned tasks, as well as the success of task negation in unlearning one task from irrelevant or contradictory tasks. Moreover, we prove the proper selection of linear coefficients for task arithmetic to achieve guaranteed generalization to out-of-domain tasks. All of our theoretical results hold for both dense-weight parameters and their low-rank approximations. Although established in a conceptual setting, our theoretical findings were validated on a practical machine unlearning task using the large language model Phi-1.5 (1.3B). 

The chemical composition and physical properties of secondary organic aerosol (SOA) generated through OH-initiated oxidation of mixtures containing β-myrcene, an acyclic monoterpene, and d-limonene, a cyclic monoterpene, were investigated to assess the extent of chemical interactions between their oxidation products. The SOA samples were prepared in an environmental smog chamber, and their composition was analyzed offline using ultra-performance liquid chromatography coupled with electrospray ionization high-resolution mass spectrometry (UPLC-ESI-HRMS). Our results suggested that SOA containing β-myrcene showed a higher proportion of oligomeric compounds with low volatility compared to SOA from d-limonene. The formula distribution and signal intensities of the mixed SOA could be accurately predicted by a linear combination of the mass spectra of SOA from individual precursors. Effects of cross-reactions were observed in the distribution of isomeric oxidation products within the mixed SOA, as evidenced by chromatographic analysis. On the whole, β-myrcene and d-limonene appear to undergo oxidation by OH largely independently from each other, with only subtle effects from cross-reactions influencing the yields of specific oxidation products.